Haptic Support for Spatial Cognitive Mapping 1 Running Head: Haptic Support for Spatial Cognitive Mapping Haptic-feedback Support for the Cognitive Mapping of Unknown Spaces by People Who Are Blind Haptic Support for Spatial Cognitive Mapping 2

Mental mapping of spaces, and of the possible paths for navigating these spaces, is essential for the development of efficient Orientation and Mobility (O&M) skills. Most of the information required for this mental mapping is gathered through the visual channel. People who are blind lack this information, and in consequence they are required to use compensatory sensorial channels and alternative exploration methods. In this study, people who are blind use a multisensory virtual environment (MVE) that provides haptic and audio feedback, to explore an unknown space. The cognitive mapping of the space with the MVE, and the subjects' ability to apply the constructed map for accomplishing tasks in the real space are examined. The results show clearly that a robust and comprehensive map is constructed, and that the exploration in the MVE contributes to the successful performance in the real space. Haptic support for spatial cognitive mapping 3 Mental mapping of spaces, and of the possible paths for navigating these spaces, is essential for the development of efficient Orientation and Mobility (O&M) skills. Most of the information required for this mental mapping is gathered through the visual channel (Lynch, 1960). People who are blind lack this information, and in consequence they are required to use compensatory sensorial channels and alternative exploration methods (Jacobson, 1993). The research reported here is based on the assumption that the supply of appropriate spatial information through compensatory sensorial channels, as an alternative to the (impaired) visual channel, may help to enhance blind people’s ability to explore unknown environments (Mioduser, in press) and to navigate in real environments. The main goals of this study were to examine: (a) the cognitive mapping process of an unknown space using a multisensory virtual environment (MVE), and (b) the application of the constructed map for performing orientation tasks in the real space. Background Research on O&M skills of people who are blind in known and unknown spaces (e.g., Passini & Proulx, 1988; Ungar, Blades & Spencer, 1996) indicates that support for the acquisition of spatial mapping and orientation skills should be supplied at two main levels: perceptual and conceptual. At the perceptual level, visual information shortage is compensated by that perceived via other senses, e.g., haptic or auditory information. For blind individuals, haptic information is commonly supplied by the white cane for low-resolution scanning of the immediate surroundings, by palms and fingers for fine recognition of object form, texture and location, and by the feet regarding navigational surface information. The auditory channel supplies complementary information about events, the presence of other people (or machines or animals) in the environment, or estimates of distances within a space (Hill, et al., 1993). Haptic support for spatial cognitive mapping 4 As for the conceptual level, the focus is on supporting the development of appropriate strategies for an efficient mapping of the space and the generation of navigation paths. Research indicates that people use two main spatial strategies: route and map strategies. Route strategy is based on linear recognition of spatial features, while map strategy is holistic and encompasses multiple perspectives of the target space (Fletcher, 1980; Kitchin & Jacobson, 1997). Research shows that people who are blind use mainly route strategy when recognizing and navigating new spaces (Fletcher, 1980). Advanced computer technologies, such as haptic-feedback technology, offer new possibilities for supplying both perceptual and conceptual support in rehabilitation and learning environments for sighted people (Giess, Evers, & Meinzer, 1998; Gorman, et al., 1998; Schultheis & Rizzo, 2001), and people who are blind as well (Lahav, 2003; Semwal & EvansKamp, 2000). The research reported in this paper aimed to examine the contribution of the work with an haptic-based Multisensory Virtual Environment (MVE) to the exploration process of an unknown space by blind subjects. The main research questions of this study were: (a) What strategies and processes are used by people who are blind for exploring an unknown space in the MVE? (b) What structural components and relationships are included in the cognitive map constructed by people who are blind who explored the unknown space in the MVE? (c) How does the (virtually) constructed cognitive map contribute to the blind person's orientation and mobility in the real space? Haptic support for spatial cognitive mapping 5 The Virtual Learning Environment For the study we developed a MVE modeling real spaces, which the user interacts with using a Microsoft SideWinder Force Feedback Joystick (FFJ). By using the FFJ the user can move within the MVE, and feel the objects’ texture, location and size. This system comprises two modes of operation: developer and learning modes. The core component of the developer mode is the MVE editor. This module includes three tools: 3D environment builder used to define the physical characteristics of the space such as type and the size of components (e.g., doors, windows, furniture pieces) and their location; Haptic feedback editor, used to attach force-feedback effects to all MVE s’ components; and Audio feedback editor to attach to the objects auditory information (e.g., its name, tapping or bumping sounds, alerts when corners are reached, footsteps). The sound interval of the footsteps indicates the speed of the navigation, and the user’s stride-length is the benchmark for distance in the virtual scene. The learning mode, within which the learner works, consists of the simulated space to be navigated by the users using the FFJ (Figure 1), and additional features that serve teachers during and after each learning session. For example, on-screen monitors present real-time information on the user’s navigation performance (e.g., position, or objects already reached). An additional feature allows the teacher to record the subject’s navigation path, and replay it to analyze and evaluate her performance (Figure 2). ------------------------------------------------------------Insert Figure 1 and 2 about here ------------------------------------------------------------Haptic support for spatial cognitive mapping 6 Method Subjects The subjects for the study were 21 people who are blind (11 congenitally blind, 10 late blind). Eleven subjects were females and ten males ranging in age from 12 to 70. Fifteen subjects were adults and six were teenager. The subjects were selected on the basis of the following seven criteria: total blindness; at least 12 years old; not multi-handicapped; received orientation and mobility (O&M) training; Hebrew speakers; onset of blindness at least two years prior to the experimental period and comfortable with the use of computers. All the subjects reported of none previous experience with MVE or FFJ. Each subject completed individually a questionnaire on O&M issues, to evaluate the participants’ initial O&M skills. The results showed no differences in initial ability among participants. Variables The three independent variables in this study were the age of vision loss, gender and subjects’ age. Three groups of dependent variables were defined, concerning (a) the exploration process of the unknown space in the MVE, (b) the construction of a cognitive map of the explored space, and (c) the subjects' performance in orientation tasks in the real space. Six variables were considered regarding the exploration process: Exploration strategies; frequency of use of the strategies; sequence of use; total distance traversed; number and duration of pauses during the exploration; total duration of the exploration. Fourteen variables concerned the construction of the cognitive map: eight related to the cognitive map's structural components (e.g., room’s size, room’s shape, inner-room objects’ Haptic support for spatial cognitive mapping 7 location); three related to spatial relationships estimation (e.g., creation of references and landmarks, distances estimation); three related to the cognitive map construction process (e.g., spatial strategies, spatial templates). Six more variables related to the performance of orientation tasks in the real space: level of success; performance strategies; navigation path; total distance traversed; total time spent on task; number and type of pauses. Research instruments Three main instruments served for the implementation of the study: The unknown space, real and simulated The real space was a room of 54 square meters with three doors, six windows and two columns. There were seven objects in the room, five of them attached to the walls and two placed in the inner space (see Figure 3). This space was virtually represented in the computer environment (see Figure 1). ------------------------------------------------------------Insert Figure 3 about here ------------------------------------------------------------Exploration task each participant was asked individually to explore the MVE, without time limitations. The experimenters informed the participants that they would be asked to describe the room and its components at the end of their exploration. Orientation tasks – each participant was asked to perform two orientation tasks in the real space: a target-object task and a perspective-taking task. In the target-object task, the subjects were asked to find an object in the space (e.g., ‘Reach and identify the object located upon the large box’). In the perspective-taking task, the subjects entered the room at a different entrance and were asked to find an object in it (e.g., ‘Find the cylinder’). Haptic support for spatial cognitive mapping 8 In addition a set of six instruments was developed for the collection of quantitative and qualitative data: O&M questionnaire comprising 46 questions concerning the participants O&M ability indoors and outdoors, in known and unknown environments. Most of the questions were taken from O&M rehabilitation evaluation instruments (e.g., Dodson-Burk & Hill, 1989; Sonn, Tornquist & Svensson, 1999). Observations the participants' exploration and task performance in the real space were video-recorded. Open interview after the exploration task the participants were asked to describe verbally the space. This open interview was video-recorded and transcribed. Modeling-kit – after describing verbally the space, the subjects were asked to construct a physical model of it. Modeling kits were already used in previous research (e.g., Kitchin and Jacobson, 1997; Passini and Proulx,1988). Ours (Figure 4) comprised three alternative structures for the rooms’ envelope and eight objects, five corresponding to these in the real environment and three distracting objects (all in three different sizes in relation to the original). The building blocks were labeled in Braille. All physical models constructed by the subjects were digitally